This invention relates generally to data storage systems and methods, and more particularly to persistent non-volatile flash memory database storage systems and methods.
The read/write performance of persistence semiconductor storage, such as flash memory, is increasing to the point where it may soon replace internal memory in some applications. The expectation is that over time non-volatile flash memory will become as fast as DRAM. This will open a number of unique and different applications for flash memory in storage systems such as databases. However, there are a number of different problems that must be addressed before flash memory can be used efficiently as a database, and this will require a different model for programming language compilers, such as Java, for example, and a dramatically different approach to storage system architectures.
Traditional Java runtime environments (JREs) do not currently have the capability of storing data objects persistently. All objects are created at runtime by constructors and stored in volatile memory (e.g., RAM). This is disadvantageous for flash memory because persistent objects in non-volatile memory need to be created, deleted and managed in an entirely different manner. An advantage to creating transient objects in volatile memory is that non-referenced objects are automatically cleared out when the application shuts down or when they are swept out by a garbage collection process, which automatically frees memory at runtime as well. This is not the situation with flash memory where affirmative action is necessary to delete persistent objects and free memory.
Further, since the number of objects in a database can be significant, there is a problem as to how to efficiently find and retrieve an object in a flash array based on a query definition. Other problems that need to be addressed when using flash memory is how to determine an object's location so that it can be referenced, and how to delete stored objects and free memory. Finding objects in persistent memory requires different indexing approaches than those used with volatile memory. Indeed, where non-volatile memory is substituting for internal memory, it is necessary to rethink traditional architectures and technologies.
Traditionally databases serialize objects into table structures using dedicated server products, and offer other advantageous capabilities such as a transactional model, a consistent view over the data, query languages such as SQL, the capability to update individual data fields, and index structures to facilitate and speed up searches. These capabilities involve significant costs that slow software solutions. The attendant serialization/de-serialization process typically goes over various hubs and caches which leads to significant overhead on both CPU and internal memory, inefficiencies and increased storage costs. This is not appropriate for flash memory storage where objects are always persisted in memory. A different transaction model for managing objects, and different approaches to retrieving and deleting objects are needed.
It is desirable to provide new memory centric database architectures based upon persistent flash memory that address the foregoing and other problems with using flash memory for database architectures, and it is to these ends that the present invention is directed.